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f) For details of the fee and ways to pay please contact the Patent Office. 
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Head Slaved Area of Interest (HSAOI) 
using framestore demultiplexing 



The device mentioned (named Optimizer) is a device for a flight simulator display 
system, but could he for a wide variety of simulation display requirements such as air 
traffic control, driving simulation etc. 

The display device can be, but is not limited to, a large array tiled projection system 
(60 separate high resolution displays) which achieves very high resolution throughout 
the field of view of the pilot etc. The reason behind the display system 's large channel 
count is to provide very high resolution imagery everywhere in the user's field of 
view, approaching or exceeding eye limiting resolution. This is essential in certain 
training tasks, for example, spotting aircraft orientation and type at realistic 
distances in air traffic control simulation. 

This number of channels in the display can not easily be driven with a sensibly priced 
graphics computer, so the device described below is configured to take a lesser 
quantity of images (20 in this example), and "spreads them out" over the (example) 
60 channels of display. This is done by interleaving images in time, so for example, 
the computer outputs 60 new images per second on each of the 20 images, with 
different views on alternative frames. Optimizer is then instructed to freeze / unfreeze 
the appropriate channel, which provides 2 discrete images, at 30Hz each from a 
single 60Hz source. This can be extended to 4 x 15 Hz images, or 8x7.5 Hz images 
etc frm a single channel. 

The invention incorporates head tracking so that the full update rate is directly ahead 
of the user 's gaze, surrounded by a 30Hz zone, then 15Hz outside of that and so on. 
Since the low update rate is in the periphery, it is not distracting. 



The main non-limiting claims are; 

• Creating a Head Slaved Area of Interest using framestore demultiplexing. 

• Using the above claim in a flight or other type of simulator 

• Using the first claim in any type of large array display device 

• Using a high speed video switcher with an FSD to create an Area of Interest 



2. 




Background 

A Head slaved Area Of Interest (HS AOI) display system is not a new concept. Various display systems 
have been supplied where there is a head tracker which controls where a projected image ends up on a 
?P!>?™ a ! _ s f reen - T .he benefit of slaving a single (or two as below) image around inside a dome is that it 
only requires 1 or 2 Image Generator (IG) channels to cover the user's field of view. 

Typical HSAOI J ^^^° " T,a B e ry in the remaining field of view 

170° widex 160° tall 

High resolution background image 
40° wide x 30 ° tall 



In this scenario, the high resolution & low resolution images are optically coupled such that when the 
steering mechanism moves these images around according to the head tracker, both move perfectly 
together. r 7 

This system has some drawbacks; 

• The steering mechanism has to respond very quickly to head movement (low latency) 

• The steering mechanism must have high acceleration & velocity to match the head's movement 
rates. 

• The High res. field of view is quite small and the eyes can see all around into the low res 
background image since it is not eye tracked. 

• There is no imagery outside the typical 1 70 x 1 60 low res. background image, but the human FOV 
is larger around 200 wide, so there is an impression of flying down a dark tunnel. 

• The amount of light able to be sent through the optics is limited. 

What SEOS are proposing is a variation of the SEOS Optimizer (already has a patent applied for) with 
a novel way of reducing the Image Generator channel count, & hence cost. 
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Seos Displays "Griffin" is a 60 (or other) channel multiple LCD tiled back projection sphere. 




Rear Projection Projector 
Screen Housings 



In this system, the large number of channels guarantees that high resolution is available wherever the 
user looks. The resolution will ultimately be eye limiting so that the display system is as resolute as the 
human eye. 

The main problem with a 60 channel display system is what sort of computer will drive 60 channels of 
imagery. Using today's cost, a 30 pipe Silicon Graphics Onyx (which can produce 60 images of 1280 x 
1024) will cost >£1 5M, so is not feasible. 

Another solution would be to use low cost PC based image generators, one per channel. This will be 
feasible in the future, but today's PC IGs do not approach the performance of the larger Onyx type IGs. 

In order to make Griffin realisable with High End Image Generators (IG) (SGI Onyx etc), we need a 
way of putting a useful amount of high resolution imagery where needed, and lower quality 
information elsewhere in the user's FOV. 

By Head tracking, we know that the images outside of the human FOV are not required since you can't 
see them. We also know that the human visual response biases the eye's highest resolution in a small 
area in the middle of the FOV. If we had a perfect eye tracking system, we could envisage a display 
which covered only this small area (say <10 degrees) with hires imagery, but this is difficult to do. 
Using Head tracking, we need to cover a much larger area with hires imagery so that the eyes can roam 
around 8c still see high res. 

The proposal for Griffin is to use a certain amount of channels from the Onyx or equivalent to provide 
a high res, full update rate (60Hz) area which is head tracked. Outside of this high res, high update rate 
area will be a medium update rate area (30Hz), and outside of this will be low update rate imagery 
(15Hz). By using a framestore on each display channel ( the SEOS Mercator, which is already present 
to perform distortion correction), we can ask the IG to interleave different channel imagery on the same 
video output. This video output is sent to multiple display channels, which are asked to freeze/unfreeze 
to let the relevant image through to the display. 



The following description is by way of example, and is not limiting. 

Take 10 pipes of Onyx, each of which can generate 2 x 1280x 1024 images at 60Hz. A head tracker 
feeds steering information to a real-time application running on the Onyx, which contains a SEOS 
software application which knows where the display channels are relative to the eye. This application 
then controls the video switch, which can switch any input channel to any output channel in real-time. 

We want 6 channels (darkest zone in the diagram below) running at 60Hz covering the area straight 
ahead of the users gaze, so the software application will configure the video switcher accordingly. 

The next zone (surrounding the darkest zone) comprises 1 4 channels which we can allow to work at 
30Hz. To do this, we ask the application to interleave 2 charmers display angles. In this way, we get : 
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Now we use the Video switcher to feed this image to both channel 1 & 2, and send a synchronisation 
signal (the Sync Processor below). This implements a freeze Channel 2's Mercator (the framestore) on 
frame 1, and unfreeze channel 2s, then on frame 2, freeze channel l's Mercator and unfreeze channel 2 
and so on. In this way, a single video output, running at 60Hz, is being split out into two 30Hz images. 
The resolution is maintained at 1280 x 1024, simply the frame rate is dropped. 



A 1 5 Hz set of channels (4 output channels for 1 input channel) would 



be 
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A 20 channel High End IG has 20 images, which are then sent to the video switcher, which then goes to 
the 60 SEOS Mercator channels present on the input to the 60 projectors. 



1 Pipe 



1 Pipe 



1 Pipe 



1 Pipe 
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Video Switch (20 in 60 out) 



Sync 
Processor 



60 Output channels to Mercator- 



_60 Freeze_ 
Signals to 
Mercators 
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This diagram below shows the 60 channels ( one small square each) of Griffin. The darkest area is at 
60Hz, the next darkest at 30Hz, the light grey at 15Hz and the white squares have no image. The user's 
gaze is tracked so that the centre of the 60Hz area is always in front. 




= 90 X 40 @ 60Hz 
= 1 50 X 80 @ 30Hz 
= 270 x 100 @ 15Hz 



What this accomplishes is using less IG channels. Normally we need 60 to drive 60 display channels, 
but in the case above, we only need 20, which is much more affordable. The novel aspects of this are: 

• Using a framestore to implement zoned areas which have lower update rate. 

• Tracking the above area with a head tracker 

• Using a high speed video switcher to implement the movement of channels 

• Grouping the 15 Hz channels together such that adjacent 15Hz channels are synchronised to 
minimise visual artefacts caused latency. 

The system (assuming a 60 channel Griffin, it could be other configurations or channels counts) 
consists of: 4 

• An Image Generator with n channels 

• A High Speed Video Switcher with n input channels, 60 output channels 

• A Sync Processor 

• 60 SEOS Mercator distortion correction boxes attached to 60 projectors ( not necessarily LCD) 

• A head tracker 

• A real-time software application which takes the head tracker input , and decides which channels 
are covered by zone A, B & C, then configures the switcher & Sync Processor accordingly. 

Other zone sizes can be implemented, examples are given below; 




= 60 x 60 @ 60 Hz 
= 120x 100@30Hz 
« 210 x 1 00 @ 15Hz 



= 60 x 40 @ 60 Hz 
= 180x80 @30Hz 
= 240 x 100 @ 15Hz 



This system addresses a lot of the existing HS AOI systems; 

• The High Update area is much larger 

• The resolution in the surrounding zones is still very high, only the frame rate has dropped. 

• There are no expensive moving parts to move the imagery around 

• The brightness is very high since there are many projectors. 

• The entire human FOV is covered by imagery. 



Glossary 
SEOS Optimizer 

SEOS Mercator 

SEOS LUCID 
Griffin 

SEOS OptiBlend 
Design Eyepoint 
Field of View 



Image Generator 
(or IG) 

FOV 
HSAOI 



High speed, real-time image processing electronics that merges high resolution 
and low resolution imagery. 

Real-time image distortion electronics which SEOS has already developed. This 
will form part of Optimizer. 

Rear-projected ultra-high resolution display system comprising 60 projected 
images onto a small sphere. 

Technology to optically blend two or more images together into one larger and 
seamless image. 

Griffin has only one place to view the image that is correct. This is in the centre 
of the spherical screen and is known as the design eyepoint. 

The viewable area defined as angles measured from the user's eyes. For example, 
current displays may have 180° horizontally by 70° vertically. Griffin is aimed to 
be 360 ° horizontally by 145 ° vertically which totally surrounds the user. 

A graphics super-computer which can generate multiple images of a virtual world 
in real time so that the user can interact with the 3D world by for example, flying 
an aircraft around in that world. 
Field of View 

Head Slaved Area Of Interest : A head tracked high fidelity region in a larger 
display system. 




Typical HSAOI 




No imagery in the remaining field of view 
170 u wide x 160 u tall 



High resolution background image 
40° wide x 30° tall 
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A 15Hz set of channels (4 output channels for 1 input channel) would be 
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l^OS^S 1 ^ ? ^ 1° ^ 20 {n,ageS ' W ^ Ch ™ ^ SCnt t0 video switcher * which then goes to 
the 60 SEOS Mercator channels present on the input to the 60 projectors. 



1 Pipe 



1 Pipe 



1 Pipe 



1 Pipe 



1 Pipe 



1 Pipe 



) Pipe 



1 Pipe 



1 Pipe 



1 Pipe 



Video Switch (20 In 60 out) 
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= 60 x 60 @ 60 Hz 
= 120x 100 @30Hz 
= 210 x 100 @ 15Hz 
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= 60 x 40 @ 60 Hz 
= 1 80 x 80 @ 30Hz 
= 240 X 100 @ 15Hz 
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